Image authentication apparatus, method, and storage medium using registered image

ABSTRACT

An image authentication apparatus includes a registration unit, a parameter computing unit, a similarity degree calculation unit, a status acquisition unit, and a generation unit. The registration unit is configured to register an image of an object to be authenticated in a registration dictionary as a registered image. The parameter computing unit is configured to compute a parameter based on the registered image. The parameter is computed for a degree of similarity between an image of the object and the registered image. The similarity degree calculation unit is configured to calculate a degree of similarity between an image of the object and the registered image, using the parameter. The status acquisition unit is configured to acquire a registration status of the registered image. The generation unit is configured to generate a display screen including the registration status, and to output the generated display screen.

BACKGROUND Field of the Invention

One disclosed aspect of the embodiments relates to an imageauthentication technology for authenticating an object; in particular, aspecific human figure from a video image input from an apparatus such asa camera.

Description of the Related Art

There is known a human figure authentication technology for capturing animage with an image capturing apparatus such as a camera, anddetermining who is a human figure appearing in the image. The humanfigure authentication technology is applied to, for example, a criminalinvestigation using a face photograph and a search for a lost child.Cameras for capturing images usually vary in installation location andviewing angle. Therefore, the cameras vary in imaging conditions inimaging of a human figure that are, for example, a direction of a humanfigure and an illumination condition. For this reason, a human figureauthentication technology that enables accurate recognition even ifimaging conditions vary is desired.

The human figure authentication technology evaluates variations incalculated degree of similarity between an image obtained by imaging ahuman figure and an image of a human figure registered beforehand(hereinafter referred to as “registered image”), and thereby determineswhether the imaged human figure matches any one of the registered humanfigures.

H. V. Nguyen and L. Bai, Cosine similarity metric learning for faceverification, in ACCV, 2011 discusses a technology for learning aparameter for similarity degree calculation beforehand based on massdata, and calculating a similarity degree based on the learnedparameter. Because the parameter is learned beforehand through use ofthe mass data including various imaging conditions, accurate recognitioncan be performed even if imaging conditions vary.

Japanese Patent No. 4343125 discusses a technology for computing atransformation matrix for transforming a feature amount obtained from animage so that human figures included in a registered image are easilydistinguished from each other, and calculating a similarity degree in atransformed feature space. Because the transformation matrix is thuscomputed based on the registered image, human figure authenticationsuitable for distinguishing a human figure within a registered image canbe performed and therefore, authentication accuracy improves.

However, in each of the technologies described above, it is difficult toobtain a similarity degree calculation parameter suitable for aninstallation location of a camera serving as an image capturingapparatus. In the technology discussed in the paper of Nguyen et al., aninstallation location of a camera is not found beforehand, andtherefore, a technology for computing a similarity degree calculationparameter suitable for an installation location beforehand is notdiscussed. In the technology discussed in Japanese Patent No. 4343125,it is possible to compute a similarity degree calculation parametersuitable for distinguishing the human figure included in the registeredimage. However, a technology for computing a similarity degreecalculation parameter suitable for an installation location of a camerais not discussed.

In addition, in each of the technologies described above, a plurality ofregistered images captured under a plurality of conditions can be used.In this case, there is such an issue that a user cannot recognizebeforehand what kind of image needs to be prepared as a registeredimage. Specifically, there is such an issue that, in a case where theuser registers his/her own image for each of a plurality of conditions,the user cannot recognize beforehand for which condition an image isregistered and for which condition no image is registered.

In general, authentication accuracy is high in a case where imagingconditions of registered images and imaging conditions of an imagecapturing apparatus installed at an authentication location are close.However, because imaging conditions, e.g., a light amount and a lightsource position, change in time and date for image capturing, dependingon the authentication location, it is desirable to prepare registeredimages corresponding to various conditions to the extent possible. Torealize this, a user needs to remember beforehand for which condition animage is already registered, which decreases the convenience of theuser.

SUMMARY

According to an aspect of the embodiments, an image authenticationapparatus includes an image acquisition unit, a registration unit, aparameter computing unit, a similarity degree calculation unit, a statusacquisition unit, and a generation unit. The image acquisition unit isconfigured to acquire an image of an object to be authenticated from animage capturing apparatus. The registration unit is configured toregister an image of the object in a registration dictionary as aregistered image. The parameter computing unit is configured to computea parameter based on the registered image. The parameter is computed forcalculation of a degree of similarity between an image of the object andthe registered image registered in the registration dictionary. Thesimilarity degree calculation unit is configured to calculate a degreeof similarity between an image of the object and the registered imagethrough use of the parameter. The status acquisition unit is configuredto acquire a registration status of the registered image in theregistration dictionary. The generation unit is configured to generate adisplay screen including the registration status, upon receipt of aregistration request for registering the image of the object in theregistration dictionary, and to output the generated display screen to adisplay apparatus.

Further features of the disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating a schematic configuration of an imageauthentication system.

FIG. 2 is a hardware block diagram illustrating an image authenticationapparatus.

FIG. 3 is a diagram illustrating a functional configuration of anauthentication unit.

FIG. 4 illustrates a console screen to be displayed by a displayapparatus.

FIG. 5 illustrates a flowchart representing authentication processing tobe executed by the image authentication apparatus.

FIG. 6 is a diagram illustrating a local area set in an authenticationimage.

FIG. 7 illustrates a flowchart representing image authenticationprocessing.

FIG. 8 illustrates a flowchart representing similarity degreecalculation parameter computing processing.

FIG. 9 is a diagram illustrating a feature amount vector space of alocal area.

FIG. 10 is a diagram illustrating a feature amount vector space of alocal area.

FIG. 11 is a diagram illustrating a screen to be displayed for a userduring execution of the similarity degree calculation parametercomputing processing.

FIG. 12 is a diagram illustrating an example of a console screen to bedisplayed by a display image generation unit.

FIG. 13 is a diagram illustrating a video image to be output from thedisplay image generation unit to the display apparatus.

FIG. 14 illustrates a flowchart representing dictionary registrationprocessing.

FIG. 15 is a diagram illustrating a screen for confirmation of aregistration status.

FIG. 16 is a diagram illustrating a feature amount vector space where aMahalanobis matrix is used as a similarity degree calculation parameter.

FIG. 17 illustrates a flowchart representing dictionary registrationprocessing.

DESCRIPTION OF THE EMBODIMENTS

A first exemplary embodiment will be described in detail below withreference to the drawings. In the following description, a similaritydegree calculation parameter is computed for each installed camera, andauthentication is performed based on a captured image to determinewhether a human figure to be authenticated is a human figure registeredin a registration dictionary. Further, as a use case of authenticationof a human figure, an example of control for entrance and exit of a gatewill be described.

FIG. 1 is a diagram illustrating a schematic configuration of an imageauthentication system 10. In FIG. 1, a human figure to be authenticated,a gate G, and the other apparatuses are arranged. FIG. 1 illustrates astate where the human figure to be authenticated moves toward the gateG.

Specifically, the image authentication system 10 includes an imageauthentication apparatus 1, a network 2, an image capturing apparatus 3,a control apparatus 4 built in the gate G, flappers 5 provided at thegate G to control entrance of a human figure, a display apparatus 6, aninput apparatus 7, and an image capturing apparatus 8. The imageauthentication apparatus 1, the image capturing apparatus 3, and theimage capturing apparatus 8 are connected to the control apparatus 4 viathe network 2.

The image authentication apparatus 1 receives a captured image bycommunicating with each of the image capturing apparatus 3 and the imagecapturing apparatus 8 via the network 2, and identifies a human figurewithin the image by performing processing such as image authenticationprocessing. Further, the image authentication apparatus 1 performsauthentication processing for the identified human figure. Furthermore,the image authentication apparatus 1 performs authentication processingfor determining whether to permit entrance of the human figure, andoutputs an authentication result obtained by the authenticationprocessing to each of the apparatuses. The network 2 is used forcommunication between the apparatuses. In the first exemplaryembodiment, a local area network is used as the network 2, but othercommunication network may be used.

In the first exemplary embodiment, a network camera including an opticallens, an image sensor, and a communication unit is used as the imagecapturing apparatus 3. The network camera includes a camera unit and acommunication device for connection to a network. The camera unitincludes a lens and an image capturing element represented by acharge-coupled device (CCD) sensor and a complementary metal oxidesemiconductor (CMOS) sensor. In place of such a network camera, anothertype of camera having a communication function may be used. The cameraunit is not limited to a camera using light including visible light, andanother type of unit such as an infrared camera unit may be used. Animage captured by the image capturing apparatus 3 is output to the imageauthentication apparatus 1 via the network 2.

The control apparatus 4 receives an authentication result from the imageauthentication apparatus 1, and outputs a control signal to the flappers5 to control opening and closing of the flappers 5. Further, the controlapparatus 4 outputs information such as a video image for displayacquired from the image authentication apparatus 1 to the displayapparatus 6. The flappers 5 are opened and closed based on controlinformation acquired from the control apparatus 4. The display apparatus6 is a display that displays information and images acquired from theimage authentication apparatus 1. The input apparatus 7 is an inputapparatus such as a keyboard, a mouse or a touch panel. The inputapparatus 7 receives an input from a user and outputs the received inputto the image authentication apparatus 1. Alternatively, a touchpanel-type display may be provided as the display apparatus 6 so thatthe display apparatus 6 also serves the input apparatus 7.

The image capturing apparatus 8 is disposed at a gate provided at alocation different from the location of the gate G illustrated inFIG. 1. The image capturing apparatus 8 has a configuration and afunction similar to the configuration and the function of the imagecapturing apparatus 3. Therefore, the image capturing apparatus 8 alsooutputs a captured image to the image authentication apparatus 1 via thenetwork 2. The above-described gate provided at the different locationis also connected to the image authentication apparatus 1 via thenetwork 2. The number of the image capturing apparatuses is not limitedto the example illustrated in FIG. 1 and may be any number.

In the first exemplary embodiment, the apparatuses are configured asseparate apparatuses, but one apparatus may have the functions of theplurality of apparatuses described above. For example, together with thecontrol apparatus 4, the image authentication apparatus 1 may be builtin the gate G.

Hardware Configuration

FIG. 2 illustrates a hardware block diagram of the image authenticationapparatus 1. As illustrated in FIG. 2, the image authenticationapparatus 1 includes a central processing unit (CPU) 11, a read onlymemory (ROM) 12, and a random access memory (RAM) 13. The imageauthentication apparatus 1 further includes a secondary storage device14, a communication device 15, a video image output device 16, and aconnection bus 18.

The CPU 11 controls the entire image authentication apparatus 1 byloading a control program or a set of instructions stored in each of theROM 12 and the secondary storage device 14 into the RAM 13, andexecuting the loaded control program. The ROM 12 is a nonvolatile memoryand stores the control program and various parameter data. The controlprogram is executed by the CPU 11 as described above, so that processingto be described below is implemented. The operations to be performed bythe CPU 11 include at least the operations described in the followingand the flowcharts. The RAM 13 is a volatile memory, and temporarilystores images and the control program as well as execution results ofthe control program. The secondary storage device 14 is a rewritablesecondary storage device such as a hard disk or a flash memory, andstores images received from the image capturing apparatuses 3 and 8 viathe communication device 15. The secondary storage device 14 also storesthe control program, and information such as various setting contentsand processing results. These pieces of information are output to theRAM 13, and used by the CPU 11 for the execution of the program.

The communication device 15 is a wire communication unit, and performscommunication with various devices via the network 2. The communicationdevice 15 may be a wireless communication unit. The video image outputdevice 16 generates a video image based on an image or an executionresult of the control program that are acquired from the RAM 13, andoutputs the generated video image to the display apparatus 6 via thecommunication device 15. The connection bus 18 connects the devicesincluded in the image authentication apparatus 1 and thereby performsdata communication between the devices.

In the first exemplary embodiment, the CPU 11 of the imageauthentication apparatus 1 executes a computer program for implementingprocessing to be described below so that the processing is implementedby software. However, part or all of the processing of the imageauthentication apparatus 1 may be implemented by hardware. A device suchas a dedicated circuit (an application-specific integrated circuit(ASIC)) or a processor (such as a reconfigurable processor or a digitalsignal processor (DSP)) can be used as hardware.

Software describing the processing to be described below may be acquiredvia a network or a computer readable storage medium, and then executedby a processing device (a CPU or a processor) of an apparatus such as apersonal computer.

Functional Configuration

FIG. 3 is a diagram illustrating a functional configuration of anauthentication unit 100 formed by the control program to be executed bythe CPU 11 of the image authentication apparatus 1. The authenticationunit 100 includes an authentication image acquisition unit 101, a localarea setting unit 102, a feature extraction unit 103, an image groupsetting unit 104, a dictionary registration unit 105, a similaritydegree calculation parameter computing unit 106, a similarity degreecalculation parameter storage unit 107, and a similarity degreecalculation unit 108. The authentication unit 100 further includes animage authentication unit 109, a display image generation unit 110, anda registration status acquisition unit 111.

The authentication image acquisition unit 101 acquires an image capturedby the image capturing apparatus 3 or the image capturing apparatus 8,acquires an authentication image based on the acquired captured image,and outputs the authentication image to the local area setting unit 102.The authentication image is an image in which an object to beauthenticated appears. In the first exemplary embodiment, theauthentication image is an image in which the face of a human figureappears. The local area setting unit 102 extracts a feature point suchas the eye or nose of the face of the human figure and informationindicating the coordinates of the feature point, from the authenticationimage acquired from the authentication image acquisition unit 101. Thelocal area setting unit 102 sets a local area based on the extractedfeature point. The local area setting unit 102 outputs the local areatogether with the authentication image to the feature extraction unit103.

The feature extraction unit 103 extracts a feature amount based on theauthentication image and the local area acquired from the local areasetting unit 102, and outputs the extracted feature amount to the imagegroup setting unit 104. In the first exemplary embodiment, the featureamount is extracted for each local area. The image group setting unit104 classifies images into groups based on the feature amounts acquiredfrom the feature extraction unit 103, determines which image belongs towhich group, and sets the result of the determination. The set group isstored into the secondary storage device 14 in association with thefeature amount, and is output to the similarity degree calculation unit108, the dictionary registration unit 105, or the similarity degreecalculation parameter computing unit 106.

The dictionary registration unit 105 stores a registration dictionaryincluding the authentication image and the feature amount acquired fromthe image group setting unit 104, into the secondary storage device 14.The registration dictionary is data in which the authentication image,the feature amount, a person identification (ID) (such as the name of aperson, in general) corresponding to the feature amount, and a thumbnailimage for display are associated with one another. The method foracquiring the person ID will be described below. The registrationdictionary may include data of a plurality of human figures, and mayinclude a plurality of pieces of data for one human figure. Further, theregistration dictionary is output to the similarity degree calculationparameter computing unit 106, the similarity degree calculation unit108, or the registration status acquisition unit 111, in response to arequest. The thumbnail image of the registration dictionary is alsooutput to the display image generation unit 110 in response to arequest. The registration status is information including the number ofregistered human figures and the number of pieces of data for a certainhuman figure. As will be described below, in another form, a pluralitypieces of registered data may be classified into specific groups and aregistration status may be calculated for each of the groups.

The similarity degree calculation parameter computing unit 106 computesa similarity degree calculation parameter, based on the feature amountacquired from the image group setting unit 104, or the feature amount ofthe registration dictionary acquired from the dictionary registrationunit 105, and outputs the calculated similarity degree calculationparameter to the similarity degree calculation parameter storage unit107. The similarity degree calculation parameter storage unit 107 storesthe similarity degree calculation parameter acquired from the similaritydegree calculation parameter computing unit 106 into the secondarystorage device 14. The stored similarity degree calculation parameter isoutput to the similarity degree calculation unit 108, in response to arequest.

The similarity degree calculation unit 108 calculates the degree ofsimilarity between the feature amount determined from the feature pointsuch as the eye or nose of the face of the human figure and acquiredfrom the image group setting unit 104, and the registration dictionaryacquired from the dictionary registration unit 105, based on thesimilarity degree calculation parameter acquired from the similaritydegree calculation parameter storage unit 107. The calculated degree ofsimilarity is output to the image authentication unit 109. Based on thedegree of similarity acquired from the similarity degree calculationunit 108, the image authentication unit 109 performs imageauthentication, and outputs an authentication result of the imageauthentication to the display image generation unit 110. As describedabove, the authentication result represents to which ID in theregistration dictionary the acquired human figure in the authenticationimage corresponds, or whether a corresponding person ID is present. Theauthentication result is output to the display image generation unit110.

The display image generation unit 110 generates a display image, usingthe authentication result acquired from the image authentication unit109, the dictionary acquired from the dictionary registration unit 105,and the registration status acquired from the registration statusacquisition unit 111. The generated display image is transmitted to thedisplay apparatus 6 via the communication device 15 and displayed by thedisplay apparatus 6.

Overall Flow

Broadly speaking, similarity degree calculation parameter computingprocessing, authentication processing, and dictionary registrationprocessing are performed in the first exemplary embodiment. In thesimilarity degree calculation parameter computing processing, asimilarity degree calculation parameter corresponding to each of theimage capturing apparatus 3 and the image capturing apparatus 8 that areinstalled is computed. In the authentication processing, authenticationis performed to determine whether an image captured by each of the imagecapturing apparatus 3 and the image capturing apparatus 8 corresponds toa human figure registered beforehand, and the opening and closing of thegate G is controlled based on the result of the authentication. In thedictionary registration processing, processing for adding a human figureto be permitted to enter and exit to a registration dictionary isperformed.

In the following description, the image capturing apparatus 3 and theimage capturing apparatus 8 are already installed at the time when eachprocessing starts, and the image authentication apparatus 1 is in astate where the image authentication apparatus 1 can immediately acquirean image to start processing to be described below.

FIG. 4 illustrates an example of a console screen to be displayed by thedisplay apparatus 6 of the image authentication system 10 in the firstexemplary embodiment. As illustrated in FIG. 4, a menu that includes“authentication/dictionary registration mode”, “adjustment mode”,“setting”, and “exit” is displayed in the console screen, and whichprocessing is to be executed can be selected. In general, anadministrator who operates the image authentication apparatus 1(hereinafter simply referred to as “administrator”) selects theprocessing from the menu via the input apparatus 7. In the illustratedexample, the processing to be described below is executed by selectionof either “authentication/dictionary registration mode” or “adjustmentmode”. Various settings of the monitoring system of the first exemplaryembodiment can be changed by selection of “setting”. The operation ofthe image authentication system 10 of the first exemplary embodimentends when “exit” is selected.

Authentication Processing

First, the authentication processing will be described. In theauthentication processing, the image authentication apparatus 1 of thefirst exemplary embodiment controls the gate G by performing imageauthentication while obtaining image data of a human figure throughimage capturing by the image capturing apparatus 3. The processing isexecuted by selection of “authentication/dictionary registration mode”from the console screen in FIG. 4. The authentication processing is alsoperformed in the image capturing apparatus 8 in parallel, but theoperation of the image capturing apparatus 8 is similar to the operationof the image capturing apparatus 3 and therefore will not be described.During the authentication processing, the image capturing apparatuses 3and 8 each continuously perform image capturing. Further, theauthentication processing proceeds based on an input acquired via theinput apparatus 7. The input is each of a “termination instruction” forterminating the authentication processing and an “authenticationinstruction” for continuously executing authentication. These two inputscan be received anytime during the authentication processing.

FIG. 5 illustrates a flowchart representing the authenticationprocessing to be executed by the image authentication apparatus 1.Actual processing will be described below in detail with reference tothe drawing. Unless otherwise specified, the CPU 11 executes each stepusing the unit such as the authentication image acquisition unit 101.

In step S1011, the authentication image acquisition unit 101 determineswhether the “termination instruction” is input. If the “terminationinstruction” is input (YES in step S1011), the processing flow ends. Ifthe “termination instruction” is not input (NO in step S1011), theprocessing proceeds to step S1013. In step S1013, the authenticationimage acquisition unit 101 acquires an authentication image from theimage capturing apparatus 3. As described above, the authenticationimage is an image in which the face of a human figure appears. Theauthentication image acquisition unit 101 acquires a captured image fromthe image capturing apparatus 3, detects the face of a human figure inthe captured image, and clips the detected face of the human figure, sothat the authentication image acquisition unit 101 can acquire theauthentication image. The captured image may be recorded in thesecondary storage device 14, and the authentication image acquisitionunit 101 may read out the captured image from the secondary storagedevice 14 as necessary in the process. Any conventional technology canbe used for the detection of the face of the human figure from thecaptured image. Such a technology is discussed in “P. Viola and M.Jones, “Robust real-time face detection”, pp. 747, Eighth InternationalConference on Computer Vision (ICCV′—Volume 2, 2001”.

A face image of a user is clipped from the captured image, based on thecoordinates of the face detected by the authentication image acquisitionunit 101, on the image. In the first exemplary embodiment, imagenormalization is performed to stabilize in-plane rotation of the facewith respect to an image plane. For example, processing such as rotatingthe image so that a straight line connecting both eyes of the face ishorizontal to the image may be performed. In the first exemplaryembodiment, the entire range of the image is used as a processingtarget, but an image range may be specified beforehand and only a facewithin the range may be used as the processing target.

Next, in step S1014, the local area setting unit 102 sets a local areawith respect to all authentication images. In the first exemplaryembodiment, first, the local area setting unit 102 detects acharacteristic area, e.g., an organ position of the face such as theposition of an eye or nose, and sets the local area based on thedetected organ position.

FIG. 6 is a diagram schematically illustrating the local area set in theauthentication image. As illustrated in FIG. 6, the right eye is set asa local area 201, the left eye is set as a local area 202, and the mouthis set as a local area 203, with respect to a face image serving as theauthentication image.

In the first exemplary embodiment, a rectangular area having thedetected organ position in the center is set as the local area, but anyother method may be used as the method for setting the local area.Alternatively, a technique not using the rectangular area having theorgan position in the center can be adopted. For example, an area basedon a position away from the organ position by a predetermined number ofpixels may be set. Alternatively, instead of the rectangular area, acircular area may be set. A predetermined area of a face image may beset beforehand without reference to the detected organ position. Thenumber of the local areas is not limited to these examples, and aplurality of local areas may be set for one organ position. Anyconventional technology can be used for the method for detecting theorgan position. For example, a technology discussed in Japanese PatentApplication Laid-Open No. 2009-211177 can be used.

In step S1015, the feature extraction unit 103 extracts a feature amountfrom each of the local areas of the authentication image. A conventionaltechnology can be used in the method for extracting the feature amount.For example, a Local Binary Pattern (LBP) feature amount can be used.Further, a Histogram of Oriented Gradients (HOG) feature amount, or aScale-Invariant Feature Transform (SIFT) feature amount may be used.Alternatively, a feature amount in which these feature amounts are mixedmay be used. Furthermore, a feature amount extracted based on aconvolutional neural network (CNN) learned beforehand may be used.Further, the extracted feature amount may be subjected to dimensionreduction through use of an analysis such as a principal componentanalysis (PCA).

In step S1016, the image group setting unit 104 defines a group for afeature amount beforehand, and determines to which group the acquiredfeature amount belongs and sets the result of the determination. Towhich group the acquired feature amount belongs is determined based on avariable factor of the image. Here, images are classified into groupsbased on time periods in which the authentication image is captured,i.e., certain lengths of time such as morning, afternoon, and night. Anillumination condition of an image varies depending on the time period,in a place such as an outdoor location, an indoor location, or adoorway. Therefore, a group can be set for each illumination condition,by defining the group based on a predetermined condition, e.g., theabove-described time period. In this way, the group can be set for eachvariable factor. The group may not be defined beforehand, and can bedynamically set by a method such as an existing clustering method, basedon the acquired image.

In step S1017, the dictionary registration unit 105 selects aregistration dictionary corresponding to the feature amount extracted instep S1015 among registration dictionaries, based on the informationabout the set group. The registration dictionaries are classified into aplurality of groups, and the dictionary registration unit 105 selects aregistration dictionary in a group that is the same as the group towhich the feature amount extracted in step S1015 belongs. In a casewhere there is no registration dictionary in the same group, aregistration dictionary of another group is used as a substitute. Thegroup of the registration dictionary will be described in detail below.Next, in step S1018, the image authentication processing is performed.The image authentication processing will be described below.

In step S1019, after the image authentication processing is performed instep S1018, the control apparatus 4 controls opening and closing of thegate G based on the authentication result acquired from the imageauthentication unit 109. In a case where a human figure corresponding tothe feature amount matches any one of human figures registered in theregistration dictionary, the control apparatus 4 opens the gate G andkeeps the gate G closed otherwise. The CPU 11 executes step S1011 againto repeat the above-described steps, until the “termination instruction”is input so that the determination in step S1011 results in YES. Theauthentication processing is thus executed. Next, the imageauthentication processing in step S1018 will be described in detail.FIG. 7 illustrates a flowchart representing the image authenticationprocessing in the first exemplary embodiment.

In step S1021, the similarity degree calculation unit 108 acquires theregistration dictionary selected in the manner described above, from thedictionary registration unit 105. As described above, the registrationdictionary is a dictionary in which data indicating a human figure to beauthenticated is registered beforehand. Here, the registration iscompleted beforehand. The registration processing will be describedbelow. Next, in step S1022, the similarity degree calculation unit 108acquires a similarity degree calculation parameter from the similaritydegree calculation parameter storage unit 107. The similarity degreecalculation parameter is computed beforehand and stored in thesimilarity degree calculation parameter storage unit 107. In theprocess, the similarity degree calculation unit 108 selects a similaritydegree calculation parameter corresponding to the group set in stepS1016. In a case where there is no corresponding similarity degreecalculation parameter, another similarity degree calculation parameteris selected. The method for computing the similarity degree calculationparameter will be described below.

In step S1023, the similarity degree calculation unit 108 acquires afeature amount of an authentication target from the image group settingunit 104. As described above, the feature amount to be acquired in theprocess is at least one or more feature amounts acquired from aplurality of authentication images. One feature amount corresponds tothe face of one human figure. Further, in the first exemplaryembodiment, because the feature amount is extracted for each of aplurality of local areas for one face, one feature amount includes thefeature amount of each of the plurality of local areas.

In step S1024, the similarity degree calculation unit 108 calculates alocal similarity degree, for the combination of each of all the featureamounts of the authentication target and the registration dictionary.The local similarity degree is a similarity degree calculated for eachof the local areas. As described above, because the feature amount ofthe authentication target and the registration dictionary correspond tothe feature amount extracted for each of the local areas, the similaritydegree is calculated for each of the local areas. The method forcalculating the local similarity degree for one of the local areas willbe described below.

A conventional technology can be used for a similarity degreecalculation method. For example, the following cosine similarity degreecan be used.S=cos θ=x′·y′/|x′∥y′|  (1)

Here, “.” indicates an inner product of a vector, and “S” indicates thedegree of similarity between feature amount vectors x′ and y′. Further,the feature amount vectors x′ and y′ are expressed as follows.x′=x _(c)−μ_(c)  (2)y′=y−μ _(c)  (3)

Here, “x_(c)” indicates a feature amount vector extracted from anauthentication image captured by a camera c. Further, “y” indicates afeature amount vector registered in a registration dictionary.Furthermore, “μ_(c)” is a vector that defines an origin point for anglecalculation. In other words, equation (1) measures a cosine similaritydegree between the feature amount vectors x_(c) and y, based on an anglebetween feature amount vectors obtained by using the origin pointdefined from μ_(c). This μ_(c) is the similarity degree calculationparameter in the first exemplary embodiment. The similarity degreecalculation parameter μ_(c) can be computed by a technique such asaveraging the feature amount vector x_(c). The method for thecomputation will be described in detail below. The method forcalculating the similarity degree for the one local area is describedabove, and a similarity degree is calculated likewise for other localareas.

In step S1025, the similarity degree calculation unit 108 calculates anintegrated similarity degree based on the calculated local similaritydegree, for the combination of each of all the feature amounts of theauthentication target and the registration dictionary. The integratedsimilarity degree is a similarity degree determined by integrating aplurality of local similarity degrees for the respective local areas. Aconventional technology can be used for the method of the integration.For example, a technique using an average of local similarity degrees ora technique of selecting a maximum value for local similarity degreescan be used. Besides, a weight for each of the local areas may becomputed beforehand and the local similarity degrees may be integratedbased on the weighted mean. In the first exemplary embodiment, thecalculation of the local similarity degree in step S1024 and thecalculation of the integrated similarity degree in step S1025 that arecombined will be referred to as similarity degree calculation.

In step S1026, for all the feature amounts of the authentication target,the image authentication unit 109 determines which one or whether noneof the human figures registered in the registration dictionary matchesthe human figure corresponding to the feature amount, based on theintegrated similarity degree. Specifically, the image authenticationunit 109 determines whether the highest integrated similarity degreeamong the integrated similarity degrees of the human figures registeredin the registration dictionary is greater than a predeterminedthreshold. In a case where the highest integrated similarity degree isgreater than the threshold, the image authentication unit 109 determinesthat the human figure corresponding to the feature amount matches thehuman figure included in the registration dictionary and correspondingto the integrated similarity degree. In contrast, in a case where thehighest integrated similarity degree is less than or equal to thethreshold, the image authentication unit 109 determines that the humanfigure corresponding to the feature amount matches none of the humanfigures in the registration dictionary. Upon completion of thedetermination in step S1026, the image authentication processing in FIG.7 ends.

As described above, the registration dictionary belonging to a groupthat is the same as the group of the authentication image and thesimilarity degree calculation parameter are used, so that authenticationaccuracy improves. The image authentication processing is executed forall the authentication images acquired in step S1013 in theauthentication processing flow in FIG. 5. However, some of theauthentication images may be selected and the image authenticationprocessing may be performed for the selected authentication images. Forexample, the authentication images may be thinned out at predeterminedinterval, or clustering may be performed for the feature amountsextracted from the authentication images and the image authenticationprocessing may be performed for a feature amount selected as arepresentative from one cluster.

Similarity Degree Calculation Parameter Computing Processing

Next, processing will be described in which the image authenticationapparatus 1 of the first exemplary embodiment computes a similaritydegree calculation parameter corresponding to each of the imagecapturing apparatuses 3 and 8 that are installed. The processing isexecuted by selection of “adjustment mode” in the console screen in FIG.4. In the first exemplary embodiment, a human figure captured by each ofthe image capturing apparatuses 3 and 8 is recorded automatically, i.e.,without requiring an instruction from the user, into the secondarystorage device 14, and a feature amount is extracted. When afeature-amount collection status satisfies a predetermined condition,e.g., when a predetermined number of feature amounts are collected, asimilarity degree calculation parameter is computed.

It is desirable that the processing be executed before the execution ofthe authentication processing described above. However, in a case whereit is difficult to perform such execution, the processing may beexecuted simultaneously with the authentication processing. In thiscase, at first, the authentication processing is performed by using asimilarity degree calculation parameter prepared beforehand. Afterward,when calculation of a new similarity degree calculation parameter iscompleted, the new similarity degree calculation parameter is used.

FIG. 8 illustrates a flowchart representing the similarity degreecalculation parameter computing processing. Actual processing will bedescribed in detail below with reference to the drawing. Each of theimage capturing apparatuses 3 and 8 continues capturing images at apredetermined frame rate from the start to the end of the flow, andcontinues storing the captured images into the secondary storage device14.

In step S1061, the authentication image acquisition unit 101 selects oneimage capturing apparatus (camera) in which the computation of asimilarity degree calculation parameter is not completed. Next, in stepS1062, the authentication image acquisition unit 101 selects an imagecaptured by the camera selected in step S1061 among the images stored inthe secondary storage device 14, and acquires an authentication imagefrom the captured image. The method for acquiring the authenticationimage from the captured image is similar to the method in theauthentication processing described above and therefore will not bedescribed. In the first exemplary embodiment, the captured image storedin the secondary storage device 14 is acquired, but the captured imagemay be directly acquired from each of the image capturing apparatuses 3and 8. Local area setting in step S1063 and feature amount extraction instep S1064 are similar to the corresponding methods described above andtherefore will not be described.

In step S1065, the image group setting unit 104 sets a group withrespect to the extracted feature amount, and stores the set group andthe feature amount in association with each other into the secondarystorage device 14. Next, the image group setting unit 104 changes theprocessing based on the status of the collection of the feature amountsextracted for each group. Specifically, in step S1066, the image groupsetting unit 104 determines whether a predetermined number of featureamounts are acquired. If the number of the collected feature amounts isless than the predetermined number (NO in step S1066), the processingreturns to step S1062. If the number of the collected feature amounts issufficient for computation of the similarity degree calculationparameter (YES in step S1066), the processing proceeds to step S1067.The image group setting unit 104 thus repeats the steps for collection(extraction and storage) of the feature amounts until the number of thefeature amounts reaches the predetermined number. In this way, in thefirst exemplary embodiment, the collection of the feature amounts endswhen the number of the feature amounts reaches the predetermined number.However, another technique may be used. For example, the collection mayend when the feature amounts are collected up to a predeterminedproportion. Alternatively, authentication may be performed for matchingbetween a human figure corresponding to the feature amount and a humanfigure registered in the registration dictionary, and the collection mayend if the result of the authentication reaches a predetermined errorrate.

Afterward, in step S1067, the similarity degree calculation parametercomputing unit 106 computes a similarity degree calculation parameter,using the collected feature amounts, for each group. How the similaritydegree calculation parameter is computed in step S1067 will be describedbelow. In step S1068, the similarity degree calculation parameterstorage unit 107 stores the computed similarity degree calculationparameter. Subsequently, in step S1069, the similarity degreecalculation parameter computing unit 106 determines whether thesimilarity degree calculation parameter computing processingcorresponding to each of all the cameras is completed. If the similaritydegree calculation parameter computing processing is completed (YES instep S1069), the similarity degree calculation parameter computingprocessing ends. If there is a camera for which the similarity degreecalculation parameter computing processing is not completed (NO in stepS1069), the processing returns to step S1061.

The processing for computing the similarity degree calculation parameterin step S1067 will be described. As described above, the similaritydegree calculation parameter in the first exemplary embodiment is theorigin point of the cosine similarity degree. A method for computing asimilarity degree calculation parameter of a certain group,corresponding to a certain local area, will be described below. Anorigin point μ_(c) corresponding to a camera c is calculated byaveraging feature amount vectors x_(ci), collected for each of theinstalled cameras, as in the following equation (4).μ_(c)=(1/N)Σ^(N) _(i) _(X) _(ci)  (4)

Here, “x_(ci)” indicates an ith feature amount vector among featureamount vectors collected from an authentication image captured by thecamera c. Further, “N” is the number of samples of the collected featureamount vectors. By using the point origin μ_(c), i.e., the similaritydegree calculation parameter, a similarity degree is calculated inmonitoring/authentication processing.

FIG. 9 is a diagram illustrating a feature amount vector space of acertain local area, captured by a certain camera. FIG. 9 schematicallyillustrates an effect produced by computing an origin point. FIG. 9illustrates an origin point 211 determined beforehand and an originpoint 215 computed based on authentication images collected by theinstalled camera. Each white circle indicates a feature amount vector,and a set of feature amount vectors, i.e., a cluster, is indicated by adotted line. In the drawing, a feature amount vector cluster 212 ofauthentication images collected by the installed camera, a featureamount vector cluster 213 of a human figure A, and a feature amountvector cluster 214 of a human figure B are each illustrated as thecluster indicted by the dotted line.

As illustrated in FIG. 9, the white circles each indicating the featureamount vector are all dispersed in the first quadrant of the graph.Meanwhile, in a case where a similarity degree is calculated based on anangle between feature amount vectors as with the cosine similaritydegree, a resolving power for distinguishing feature amount vectors ishigh if the origin point is present inside a cluster formed of featureamount vectors of an authentication target.

The origin point 211 determined beforehand is present outside thecluster formed of the feature amount vectors of the authenticationtarget, and is not always present inside the cluster. For example, if anangle between a feature amount vector of the human figure A (any one ofthe feature amount vectors in the feature amount vector cluster 213) anda feature amount vector of the human figure B is measured with referenceto the origin point 211 determined beforehand, the angle is close to 0.Therefore, if a cosine similarity degree is calculated in a state wherethe origin point 211 is present outside the cluster formed of thefeature amount vectors as illustrated in FIG. 9, a high similaritydegree that is likely to result in determination of another person asthe authentic person by mistake can be calculated.

Therefore, although the origin point 211 being present inside thecluster formed of the feature amount vectors is desirable, the originpoint 211 is not always present inside the cluster because, in general,the origin point 211 is calculated based on a large number ofauthentication images prepared beforehand. The feature amount vectordistribution of the authentication images collected by the installedcamera is different from the feature amount vector distribution of theauthentication images prepared beforehand, in some cases. In particular,because the installed camera is limited in terms of viewing angle andlocation, a large number of images under a specific illuminationcondition and images including faces in a specific direction arecaptured by the installed camera, as compared with the authenticationimages prepared beforehand. In such a case, the origin point 211 islocated outside the cluster formed of the feature amount vectors asillustrated in FIG. 9 and thus, an incorrect high similarity degree iscalculated as described above. This can lead to false authentication.

Therefore, the origin point 215 computed based on the authenticationimages collected by the installed camera is used in place of the originpoint 211 computed beforehand, so that the origin point is locatedinside the feature amount vector cluster 212 of the authenticationimages collected by the installed camera. A cosine similarity degree canbe thereby calculated by using an origin point suitable forauthentication for each installation location of the camera, so that areduction in false authentication is expected.

However, the origin point computed based on the authentication imagescollected by the installed camera is not always present at a positionsuitable for authentication within a feature space. FIG. 10 is a diagramillustrating a feature amount vector space. FIG. 10 illustrates anorigin point 221 determined beforehand, a feature amount vector cluster222 of authentication images collected by an installed camera, a featureamount vector cluster 223 of a human figure C, and an origin point 224computed based on authentication images collected by an installedcamera. In this case, the origin point 221 determined beforehand is notsuitable for calculation of a cosine similarity degree, as with the caseillustrated in FIG. 9. However, the origin point 224 computed based onthe authentication images collected by the installed camera is presentinside the feature amount vector cluster 223 of the human figure C.

In this case, if a cosine similarity degree is calculated, such falseauthentication that the human figure C of the feature amount vectorcluster 223 is determined as an authentic person by mistake can occur.In this way, in a case where the computed origin point is present insidethe feature amount vector cluster of someone else, the origin point isnot suitable for authentication.

Therefore, instead of extracting one feature amount for oneauthentication image, a plurality of feature amounts is extracted bysetting a plurality of local areas, and similarity degrees of therespective feature amounts are eventually integrated. This reduces falseauthentication that occurs in a case where there is a local area inwhich the computed similarity degree calculation parameter is notsuitable for authentication. Therefore, even if a high similarity degreelikely to result in determination of someone else to be an authenticperson is obtained based on a similarity degree calculation parameterunsuitable for authentication in a certain local area, a similaritydegree calculation parameter suitable for authentication can be computedin the other local areas. Further, because these similarity degreecalculation parameters of local areas are eventually integrated, areduction in false authentication can be expected.

The plurality of feature amounts is extracted by setting the pluralityof local areas, but the plurality of feature amounts may be extracted byanother method. For example, the plurality of feature amounts may beextracted by extracting different types of feature amounts. In the firstexemplary embodiment, the average of the feature amount vectors is usedas the origin point. However, the origin point may be computed byanother method. For example, in place of a simple average of the featureamount vectors, a weighted mean may be used. In this case, the value ofa weight may be normalized such that an origin point is located inside acluster formed of feature amount vectors for authentication.

As described above, the similarity degree calculation parametercorresponding to one local area and one group is computed. Thecomputation technique is applied to all the other local areas andgroups.

FIG. 11 is a diagram illustrating a screen to be displayed for a user,during the execution of the similarity degree calculation parametercomputing processing. The user in the processing is, in general, anadministrator who operates the image authentication system 10, althoughthe user is not limited to the administrator. The display screendisplays a status of collection of feature amounts for adjusting thesimilarity degree calculation parameter. In this example, acollected-sample count indicates “OK” for all groups in the imagecapturing apparatus 3. This indicates that a sufficient number offeature amounts are collected to compute the similarity degreecalculation parameter, for all the groups. In contrast, the exampleindicates that the collected-sample count is still insufficient in thegroup of morning, for the image capturing apparatus 8. In this way, thecollection of a predetermined number of images is displayed for theuser, so that the user can visually recognize how many images still needto be collected, and thus the convenience increases.

In the first exemplary embodiment, the similarity degree calculationparameter computing processing has been described not to be completedwithout computation of the similarity degree calculation parameter.However, the similarity degree calculation parameter computingprocessing may be suspended before completion of the computation of thesimilarity degree calculation parameter. In this case, the similaritydegree calculation parameter to be used for the authenticationprocessing is prepared beforehand by another method. For example, adefault similarity degree calculation parameter prepared beforehand maybe used. Alternatively, the similarity degree calculation parameter maybe computed in a state where a predetermined number is not reached.Still alternatively, feature amounts that still need to be collected forcomputation may be collected in the background of the authenticationprocessing, and the similarity degree calculation parameter may becomputed when the predetermined number is reached. In this case, adefault similarity degree calculation parameter prepared beforehand isused until the computation of the similarity degree calculationparameter is completed, and the default similarity degree calculationparameter is replaced with the computed similarity degree calculationparameter upon completion of the computation.

Dictionary Registration Processing

Next, the dictionary registration processing for registering a humanfigure to authenticate will be described. In the first exemplaryembodiment, a case will be described where the image capturing apparatus3, i.e., a camera that is the same as the camera used in theabove-described authentication processing, acquires an authenticationimage that is an image to be registered in a dictionary. In this case,an image captured under imaging conditions (such as an illuminationcondition and a face direction) that are the same as imaging conditionsin an authentication location can be registered in a dictionary, so thatan improvement in the authentication accuracy can be expected. A camerafor dictionary registration may be installed in another location. Inthis case, however, because the imaging conditions are different fromthe imaging conditions in a monitoring and authentication location, itis necessary to be careful about whether the authentication accuracy issufficient. A case where the image capturing apparatus 8 is used issimilar and therefore will not be described.

A case where an authentication subject who enters and exits through agate performs registration will be described. A user in the processingis the authentication subject.

FIG. 12 illustrates a console screen to be displayed by the displayimage generation unit 110, in “authentication/dictionary registrationmode”. The user performs registration by selecting “face imageregistration” from a menu in the screen.

FIG. 13 is a schematic diagram illustrating an example of a video imageto be output from the display image generation unit 110 to the displayapparatus 6 in the dictionary registration processing. In an area 301 onthe left side of FIG. 13, an image acquired from the image capturingapparatus 3 is displayed as a live view. Because the live view is thusdisplayed, the user can perform dictionary registration while confirminghow the face appears, so that the convenience of the user improves. Torealize such dictionary registration, the image capturing apparatus 3 isdisposed to face in the same direction as the screen of the displayapparatus 6. The image displayed in the area 301 includes the face ofthe user, and an area 302 on the right side of FIG. 13 indicates aregistration status, examples of a registered image, and a person ID. Inthe example illustrated in FIG. 13, the registration status includes“registered”, “unregistered”, and “others” displayed for each ofmorning, afternoon, and night for each person ID. Because the uservisually recognizes the registration status, the user can confirmwhether an authentication image including the face of the user isclipped from an image of the user captured by the image capturingapparatus 3, and whether the clipped image is registered, for eachperson ID, as well as for each of morning, afternoon, and night. In alower-left part of FIG. 13, a registration button 305 and an exit button306 being displayed are illustrated. The user inputs a registrationrequest for performing the dictionary registration into the imageauthentication apparatus 1, by selecting the registration button 305, sothat the dictionary registration processing begins. The user terminatesthe dictionary registration processing by selecting the exit button 306.

The user can recognize whether the dictionary registration has beenperformed for, for example, each of a plurality of states at the time ofimage capturing by the image capturing apparatus 3, such as morning,afternoon, and night, by viewing the registration status. Therefore, theuser can perform registration by selecting a displayed state indicatingthat no dictionary registration has been performed for the user. It istherefore not necessary for the user to remember in which state theregistration has been performed and thus, the convenience improves.Moreover, because a state that needs dictionary registration isdisplayed and the user is prompted to perform the dictionaryregistration for the displayed state, the dictionary registration isselectively performed for the state that needs the dictionaryregistration. Therefore, the dictionary registration is efficientlyperformed for the state that needs the registration of an authenticationimage and thus, the authentication accuracy improves.

FIG. 14 illustrates a flowchart representing the dictionary registrationprocessing. The dictionary registration flow will be described in detailbelow with reference to FIG. 14. As described above, the live view isdisplayed in the area 301 in FIG. 13, at the start of the processing inFIG. 14. Unless otherwise specified, the CPU 11 executes the processingin FIG. 14. In step S1031, the CPU 11 determines whether a terminationinstruction is input from the user by selection of the exit button 306illustrated in the lower-left part of FIG. 13. If the terminationinstruction is input (YES in step S1031), the dictionary registrationprocessing ends. If the termination instruction is not input (NO in stepS1031), the processing proceeds to step S1032. In step S1032, the CPU 11determines whether a person ID is input. If no person ID is input fromthe user (NO in step S1032), step S1031 is executed again. If a personID is input (YES in step S1032), step S1033 to be described below isexecuted.

To avoid changing of the registration dictionary by someone else withoutpermission, it is desirable to perform authentication of an authenticperson when receiving an input of a person ID. For example, encrypteddata indicating the person ID is stored in a contactless card, and theperson can input the person ID by touching the card on a card reader, sothat the authentication can be performed. The authentication of theperson can be accurately performed unless the card of the person isstolen.

In step S1033, upon the input of the person ID, the CPU 11 displays aregistration status corresponding to the person ID, using the displayimage generation unit 110. In the process, the CPU 11 acquires theregistration status, using the registration status acquisition unit 111.The registration status is information that indicates an imageregistered for each human figure or the presence/absence of a featureamount. As described above, the area 302 illustrated in FIG. 13indicates the example of a registration status to be displayed. Asillustrated in FIG. 13, group information is displayed. Further, whetheran image is registered or unregistered for each of morning, afternoon,and night for a human figure having a person ID “0001” is displayed. Inthe example, morning, afternoon, and night are defined and displayed asthe group information, and an image is registered for morning, whereasno image is registered for each of afternoon and night.

Further, in the example illustrated in FIG. 13, the registered images ofother human figures are displayed as the examples of the registeredimage on the right side, for each of afternoon and night for which noimage is registered. Because the examples of the registered image arethus displayed, the user can recognize what kind of image is desirablefor registration, so that the convenience improves.

Next, in step S1034, the CPU 11 acquires data representing an image forregistration, using the authentication image acquisition unit 101. Inthe area 301 illustrated in FIG. 13, a face frame 303 is added to theappearing face in the previous stage. Therefore, the image is acquiredas an image for registration by the selection of the registration button305 by the user. The face frame 303 may be displayed by using atechnology of the above-described face detection or the like. Before theregistration button 305 is selected, an arrow 304 is provided tohighlight to which one of the morning, afternoon, and night groups theface of the user is to be added by selection of “registration”.

As described above, the registration dictionaries are classified intothe plurality of groups that are the three groups of morning, afternoon,and night in the first exemplary embodiment. Because the registrationdictionaries are thus classified into the groups corresponding to thedifferent imaging conditions, the similarity degree calculation betweenthe imaging conditions close to the imaging condition in theauthentication can be performed, so that an improvement in accuracy canbe expected. When the registration is performed, to which group theimage is to be added is highlighted and displayed together with theregistration status, so that the user can recognize to which group theimage currently intended to be registered belongs. In addition, the usercan visually recognize whether the image intended to be registered is anunregistered image, so that the convenience improves.

The image for registration is not necessarily an image captured by theimage capturing apparatus 3 or 8, and an captured image beforehand maybe used. Subsequent local area setting in step S1035 and feature amountextraction in step S1036 are similar to step S1014 and step S1015,respectively, in FIG. 5 described above and therefore will not bedescribed.

Next, in step S1037, the CPU 11 registers the feature amount, using thedictionary registration unit 105. In this process, the CPU 11 performsdetermination and setting of the group, using the image group settingunit 104, and registers the set group in the dictionary, in associationwith the feature amount together with the input person ID describedabove.

Upon completion of the registration, the processing returns to stepS1031.

The feature amount registered in step S1037 is used in the imageauthentication processing described with reference to step S1018 in FIG.5, and step S1021 to step S1026 in FIG. 7. In the first exemplaryembodiment, the time period defined as each of morning, afternoon, andnight is used as the variable factor of an image, and grouping isperformed based on these time periods. The registration status isacquired and the acquired registration status is displayed based on thegroups. However, the grouping may be performed based on another variablefactor. For example, the grouping may be performed based on time periodsor times of day that are different from the time periods of morning,afternoon, and night and are set beforehand. For an environment of anoutdoor location or an environment close to the outdoor location such asan indoor place covered with glass, the grouping may be performed basedon the weather, because a light amount at the time of image capturing isaffected by the weather in such an environment. For an indoorenvironment not affected by the weather, the grouping may be performedbased on control of the light amount of illumination. In these examples,the grouping can be performed based on the light amount at the time ofimage capturing.

In the first exemplary embodiment, the similarity degree calculationparameter is computed by acquiring the authentication image from theimage captured by the installed camera. Therefore, human figureauthentication can be normally implemented, even in a case where theimaging conditions are different. Because the camera is installed, theimaging conditions such as the direction of a human figure to be imagedand the illumination condition are fixed to some extent. In particular,it is desirable that, by using a fixed camera, the position, the height,and the capturing angle of the camera at the time of image capturing befixed, and the position at the time of imaging a human figure be fixed.Because the similarity degree calculation parameter dedicated to theimaging condition of the installation location of the camera is used, ahuman figure can be authenticated more accurately than in a case where asimilarity degree calculation parameter is used which is preparedbeforehand and for which a condition for the installation location of acamera is not determined.

In addition, in the dictionary registration processing, the currentregistration status is displayed for the user, by using an apparatussuch as the display apparatus 6. Therefore, the user can understand whatkind of image is desirable for registration and thus, the convenienceimproves.

In the first exemplary embodiment, the example has been described inwhich the authentication subject performs the registration in thedictionary by operating the console, but only the administrator mayperform the operation for the registration. In this case, a human figureto be registered is positioned in front of the camera, and theadministrator performs the registration processing by operating theconsole. In this case, because only the administrator performs theregistration of a human figure in the registration dictionary, thesecurity increases thanks to a reduction in risk such as tampering ofthe registration dictionary, although the convenience decreases.

In a case where the user performs the registration, additionalregistration of an image may be prompted if the gate is not opened dueto an authentication failure. For example, the additional registrationcan be audibly prompted, or a screen for promoting the additionalregistration can be displayed at the display apparatus 6 or a displayseparately installed. In the first exemplary embodiment, theregistration status is displayed in the dictionary registrationprocessing, but there is a case where the administrator needs to confirmthe registration status. For such a case, the administrator may beenabled to confirm the current registration status.

FIG. 15 is an example of a screen for confirmation of the registrationstatus to be displayed when “registration status confirmation (foradministrator)” in FIG. 12 is selected. As illustrated in FIG. 15, thegroup information (the morning, afternoon, and night groups) isdisplayed together with the person ID in the screen for confirmation. Inthe illustrated example, for a person ID “0001”, an image is registeredfor each of the morning and afternoon groups, whereas no image isregistered for the night group.

For a person ID “0002”, the same image is registered for the morning andafternoon groups, and a different image is registered for the nightgroup.

For a person ID “0003”, an image is registered for the morning group,whereas no image is registered for the afternoon and night groups.

In this example, the number of registered images is displayed in each ofthe columns of the respective morning, afternoon, and night groups. Asan example, in a display portion 701, “morning (25/30)” for the personID indicates that 30 person IDs are registered, and registration formorning has been performed for 25 out of these 30 person IDs. Theadministrator can understand which group (among the morning, afternoon,and night groups in this example) has a small number of registeredimages by confirming this display screen. In general, in a time periodwith no registered image, authentication accuracy is expected to belower than that in a time period with registered images.

Therefore, for a time period with a small number of registered images,measures such as changing the operation of the image authenticationsystem (such as positioning a guard because non-authentication easilyoccurs) can be taken, so that the convenience can be improved.

Further, the administrator may be enabled to add an image of a usercaptured beforehand to the registration dictionary. In this case, aninterface for inputting the image captured beforehand is prepared, andthe image is input together with a person ID. This makes it possible todisplay into which group the image is to fall.

In the first exemplary embodiment, the group names (morning, afternoon,and night) are displayed as the group information, but other informationmay be displayed. For example, the morning, afternoon, and night groupsmay each be more specifically defined and displayed. For instance, themorning group may be a time period of 6:00 a.m. to 11:00 a.m. In a casewhere the groups are not defined beforehand and are dynamically definedfrom an image, the groups may be provided with convenient names such asgroup 1 and group 2, and displayed.

In the first exemplary embodiment, the cosine similarity degree is usedas the similarity degree, and the origin point in the calculation of thecosine similarity degree is computed as the similarity degreecalculation parameter, but other similarity degree may be used. Forexample, as represented by the following equations, the reciprocalnumber of a Mahalanobis distance may be used as a similarity degree S,and a Mahalanobis matrix M may be computed as the similarity degreecalculation parameter.S=1/D _(M)  (5)D _(M)=√{(x _(c) −y)^(T) M _(c) ⁻¹(x _(c) −y)}  (6)

Here, x_(c) and y indicate a feature amount vector of a human figureimaged by a camera c and a feature amount vector of a registrationdictionary, respectively. FIG. 16 is a diagram illustrating a featureamount vector space in which a Mahalanobis matrix is used as thesimilarity degree calculation parameter, and illustrates an effect ofusing the Mahalanobis matrix. FIG. 16 illustrates a distribution offeature amount vectors as with FIG. 9 and FIG. 10. FIG. 16 illustrates amean vector 231 computed beforehand, a distance space 232 based on aMahalanobis matrix computed beforehand, and a mean vector 233 computedas a new value based on authentication images collected by the installedcamera. Further, FIG. 16 also illustrates a distance space 234 based ona computed new Mahalanobis matrix, a feature amount vector cluster 235of a human figure D, and a feature amount vector cluster 236 of a humanfigure E.

The distance space 232 indicates that the Mahalanobis distances from themean vector 231 computed beforehand are equal on the ellipse of thedistance space 232. In other words, in the Mahalanobis distance of thedistance space 232, a distance in a diagonal direction from a lower leftpart to an upper right part of FIG. 16 has a shorter distance scale thanin a normal Euclidean distance. This also applies to the distance space234 in which a distance from an upper left part to a lower right part ofFIG. 16 has a shorter distance scale than in a normal Euclideandistance. In this way, the distance scale computed beforehand and thedistance scale computed from the authentication images collected by theinstalled camera are different.

In FIG. 16, in the distance space 232 based on the Mahalanobis matrixcomputed beforehand, the distance from the lower left part to the upperright part of FIG. 16 has the short distance scale. Therefore, theMahalanobis distance between the feature amount vector of the humanfigure D (any one point within the feature amount vector cluster 235 ofthe human figure D) and the feature amount vector of the human figure Eare close, and thus, both can be determined as the same human figure.However, on the contrary, in the distance space 234 based on thecomputed new Mahalanobis matrix, the distance between the feature amountvector of the human figure D and the feature amount vector of the humanfigure E is long and thus, both are highly likely to be determined asdifferent persons. In this way, the technique of the first exemplaryembodiment can also be applied to the technique of setting thesimilarity degree calculation parameter in preliminary learning, such asbased on the Mahalanobis distance.

In addition, in the first exemplary embodiment, the authenticationtarget is the face of a human figure, and the authentication image is animage where a face appears. However, the authentication subject is notlimited to the face, and may be an image where a human body or anotherobject appears. As described above, in the first exemplary embodiment,the authentication is performed by using the authentication scheme inwhich the system administrator registers the authentication subject inthe dictionary and which uses the local area.

In the first exemplary embodiment, the example has been described inwhich the similarity degree calculation parameter is computed by usingthe feature amounts collected from the captured image. In a secondexemplary embodiment, an example in which a similarity degreecalculation parameter is computed in dictionary registration processingwill be described. In the following description, processing similar tothe processing in the first exemplary embodiment will not be described.

First, an example will be described in which, as with the firstexemplary embodiment, a cosine similarity degree is used as a similaritydegree in computing the similarity degree calculation parameter in thedictionary registration processing. Monitoring/authentication processingand similarity degree calculation parameter computing processing aresimilar to those in the first exemplary embodiment, and therefore willnot be described, and only the dictionary registration processing willbe described. FIG. 17 illustrates a flowchart representing thedictionary registration processing in the second exemplary embodiment.

In step S1040, a CPU 11 determines whether a termination instructionfrom a user is input. If the termination instruction is input (YES instep S1040), this dictionary registration processing ends. If thetermination instruction is not input (NO in step S1040), the processingproceeds to step S1041. In step S1041, the CPU 11 acquires an image forregistration. Subsequently, the CPU 11 sets a local area in step S1042,and then extracts a feature amount in step S1043. Further, in stepS1044, the CPU 11 registers the feature amount. These steps S1041 toS1044 are similar to steps S1034 to S1037, respectively, in FIG. 14 inthe first exemplary embodiment, and therefore will not be described indetail. Next, in step S1045, the CPU 11 updates a similarity degreecalculation parameter through use of a registration dictionary acquiredfrom a dictionary registration unit 105, using a similarity degreecalculation parameter computing unit 106.

Assume that a similarity degree calculation parameter computed in thesimilarity degree calculation parameter computing processing beforeparameter update, i.e., a mean vector of feature amounts collected by acamera at an installation location, is σ₀, and a mean vector of all thefeature amounts of the registration dictionary is σ_(r). In this case, asimilarity degree calculation parameter σ after the update is expressedas follows.σ=w ₀σ₀ +w _(r)σ_(r)  (7)

In this equation, w₀ and w_(r) are weight coefficients for therespective similarity degree calculation parameters. In general, ascompared with the number of feature amounts that can be collected by aninstalled camera, the number of feature amounts of a registrationdictionary is small. Therefore, if an average is determined from thefeature amounts of both sides, the influence of the feature amountscollected by the installed camera is large.

Therefore, as in the second exemplary embodiment, the respectivecalculated mean vectors are added by using predetermined weights, sothat the extent of the influence of the mean vector determined based onthe registration dictionary can be adjusted. The mean vector σ₀ may becomputed beforehand. Alternatively, all the feature amounts collected bythe camera at the installation location may be stored in a secondarystorage device 14, and the mean vector σ₀ may be computed when theequation (7) is computed.

Next, in step S1046, the CPU 11 stores the similarity degree calculationparameter σ computed based on the equation (7) into the secondarystorage device 14, using a similarity degree calculation parameterstorage unit 107. In the second exemplary embodiment, each time a newfeature amount is registered into the registration dictionary, thesimilarity degree calculation parameter is computed, but the similaritydegree calculation parameter is not necessarily computed each time. Forexample, the similarity degree calculation parameter may be computedwhen a predetermined number of new feature amounts are registered in theregistration dictionary.

In addition, in the second exemplary embodiment, the mean vector σ_(r)of all the feature amounts of the registration dictionary is computedwith regard to equation (7), but may be computed by using anothermethod. For example, first, a mean vector may be computed for each ofhuman figures in the registration dictionary, and then the average ofthe mean vectors of the respective human figures may be calculated. Thiscan inhibit the influence of a human figure having a large number ofregistered images from becoming excessively large, in a case where thereis a large difference in the number of registered images between humanfigures.

As described above, in the second exemplary embodiment, not only thefeature amounts collected from the captured image obtained by theinstalled camera as in the first exemplary embodiment, but also thefeature amounts of the registration dictionary are reflected on thecomputation of the similarity degree calculation parameter. By usingthis technique, the similarity degree calculation parameter moresuitable for the imaging conditions of the registered image can becomputed.

As described above, according to each of the exemplary embodiments, in acase where the user performs the image registration, the userunderstands what kind of image is desirable for registration, so thatthe convenience in image registration improves. Further, the originpoint suitable for authentication is used for each installation locationof the camera and the plurality of local areas are set so that theorigin point of the feature amount vectors is located inside the clusterof the feature amount vectors. Therefore, false authentication can bereduced.

The disclosure is not limited to each of the exemplary embodimentsdescribed above. For example, in each of the exemplary embodiments, thedescription is provided by using the case of the human face recognitionas an example, but the disclosure is applicable to another object suchas a pupil or a fingerprint of a human. In this case, while the localarea is set based on the position of an organ such as the eye or nose ofa face in the exemplary embodiments described above, a local area is setbased on a characteristic area such as an iris and a feature amount isextracted.

Other Embodiments

Embodiment(s) of the disclosure can also be realized by a computer of asystem or apparatus that reads out and executes computer executableinstructions (e.g., one or more programs) recorded on a storage medium(which may also be referred to more fully as a ‘non-transitorycomputer-readable storage medium’) to perform the functions of one ormore of the above-described embodiment(s) and/or that includes one ormore circuits (e.g., application specific integrated circuit (ASIC)) forperforming the functions of one or more of the above-describedembodiment(s), and by a method performed by the computer of the systemor apparatus by, for example, reading out and executing the computerexecutable instructions from the storage medium to perform the functionsof one or more of the above-described embodiment(s) and/or controllingthe one or more circuits to perform the functions of one or more of theabove-described embodiment(s). The computer may comprise one or moreprocessors (e.g., central processing unit (CPU), micro processing unit(MPU)) and may include a network of separate computers or separateprocessors to read out and execute the computer executable instructions.The computer executable instructions may be provided to the computer,for example, from a network or the storage medium. The storage mediummay include, for example, one or more of a hard disk, a random-accessmemory (RAM), a read only memory (ROM), a storage of distributedcomputing systems, an optical disk (such as a compact disc (CD), digitalversatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, amemory card, and the like.

While the disclosure has been described with reference to exemplaryembodiments, it is to be understood that the disclosure is not limitedto the disclosed exemplary embodiments. The scope of the followingclaims is to be accorded the broadest interpretation so as to encompassall such modifications and equivalent structures and functions.

This application claims the benefit of Japanese Patent Application No.2018-075350, filed Apr. 10, 2018, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image authentication apparatus comprising: animage acquisition unit configured to acquire an image of an object to beauthenticated from an image capturing apparatus; a registration unitconfigured to register the image of the object in a registrationdictionary as a registered image; a setting unit configured to set agroup to which the registered image is to be assigned; an output unitconfigured to output a registration status of an object to be registeredfor each group based on the group of the registered image set by thesetting unit; a determination unit configured to determine, in a casewhere there is a group in which a number of the registered images set bythe setting unit is larger than a predetermined number, a parameter forcalculating a degree of similarity between an input image and theregistered image registered in the registration dictionary; and acalculation unit configured to calculate the degree of similaritybetween the input image and the registered image registered in theregistration dictionary using the determined parameter.
 2. The imageauthentication apparatus according to claim 1, wherein the parameter isdetermined for each group set by the determination unit.
 3. The imageauthentication apparatus according to claim 1, wherein the output unitoutputs a display screen including the registration status, for eachgroup set by the group setting unit.
 4. The image authenticationapparatus according to claim 3, wherein the output unit outputs thedisplay screen to include a group to which no image is assigned, or agroup in which a number of assigned images is less than that in anothergroup.
 5. The image authentication apparatus according to claim 1,wherein the registration status includes information indicating presenceor absence of the registered image for each group set by the settingunit.
 6. The image authentication apparatus according to claim 1,wherein the setting unit dynamically sets the group based on the imageacquired by the image acquisition unit.
 7. The image authenticationapparatus according to claim 1, wherein the setting unit sets the groupbased on a variable factor of the image acquired by the imageacquisition unit.
 8. The image authentication apparatus according toclaim 7, wherein the variable factor of the image is an illuminationcondition of the image acquired by the image acquisition unit.
 9. Theimage authentication apparatus according to claim 7, wherein thevariable factor of the image is a time at which the image is acquired bythe image acquisition unit.
 10. The image authentication apparatusaccording to claim 7, wherein the variable factor of the image isweather in which the image is acquired by the image acquisition unit.11. The image authentication apparatus according to claim 1, whereinupon receipt of a registration request for registering an image of theobject in the registration dictionary, the output unit acquiresinformation about the set group from the setting unit, generates adisplay screen including the information about the group, and outputsthe generated display screen to the display apparatus.
 12. The imageauthentication apparatus according to claim 1, wherein the registrationstatus includes information indicating presence or absence of an imageregistered in the registration dictionary, for each human figure to beauthenticated.
 13. The image authentication apparatus according to claim1, wherein the output unit outputs a display screen to includeinformation representing a number of images acquired by the imageacquisition unit.
 14. The image authentication apparatus according toclaim 1, further comprising an area setting unit configured to set aplurality of local areas with respect to the image acquired by the imageacquisition unit, wherein the determination unit determines theparameter for each of the plurality of local areas set by the areasetting unit, and wherein the calculation unit calculates the degree ofsimilarity for each of the plurality of local areas set by the areasetting unit.
 15. The image authentication apparatus according to claim1, further comprising an extraction unit configured to extract a featureamount from an image acquired by the image acquisition unit, wherein thedetermination unit determines an origin point for defining an anglebetween vectors in a feature space, as the parameter, and wherein thecalculation unit calculates the degree of similarity based on an anglebetween vectors, using the computed origin point.
 16. The imageauthentication apparatus according to claim 15, wherein the origin pointis computed from an average of a plurality of feature amounts extractedby the extraction unit.
 17. The image authentication apparatus accordingto claim 1, wherein the parameter is a Mahalanobis matrix, and whereinthe calculation unit calculates the degree of similarity based on theMahalanobis matrix.
 18. An image authentication method comprising:acquiring an image of an object to be authenticated from an imagecapturing apparatus; registering the image of an object in aregistration dictionary as a registered image; setting a group to whichthe registered image is to be assigned; outputting a registration statusof an object to be registered for each group based on the group of theregistered image; determining, in a case where there is a group in whicha number of the registered images is larger than a predetermined number,a parameter for calculating a degree of similarity between an inputimage and the registered image registered in the registrationdictionary; and calculating the degree of similarity between the inputimage and the registered image registered in the registration dictionaryusing the determined parameter.
 19. A non-transitory computer-readablestorage medium that stores a computer program for causing a computerconnected to an image capturing apparatus to operate as: an imageacquisition unit configured to acquire an image of an object to beauthenticated from the image capturing apparatus; a registration unitconfigured to register the image of an object in a registrationdictionary as a registered image; a setting unit configured to set agroup to which the registered image is to be assigned; an output unitconfigured to output a registration status of an object to be registeredfor each group based on the group of the registered image set by thesetting unit; a determination unit configured to determine, in a casewhere there is a group in which a number of the registered images set bythe setting unit is larger than a predetermined number, a parameter forcalculating a degree of similarity between an input image and theregistered image registered in the registration dictionary; and acalculation unit configured to calculate the degree of similaritybetween the input image and the registered image registered in theregistration dictionary using the determined parameter.